Human-machine interaction method based on visual stimulation

ABSTRACT

A human-machine interaction method based on visual stimulations. The method can be applied to multiple new technical fields of display, which comprise but are not limited to the fields of virtual reality (VR), augmented reality (AR), mixed reality (MR), holographic projection and glasses-free 3D. The system consists of three parts: a human body biological collection apparatus, a software client for human-machine interaction and a display terminal. Input ports of the software client are connected to a human body physiological signal collection device (in a wired or wireless manner); a user wears the collection device, and communication ports of the client are respectively connected to communication ports of a display module by means of a multichannel communication module. Firstly, the system is initialized, and then starts to run based on a control method of visual stimulations (an object flicker or distortion). If a target is a text input target, an interface is switched to a text input interface, and texts are inputted by using a physiological signal detection algorithm for a human body. If the target is not a text input target, the type of information is determined by using a detection algorithm of a specific physiological signal and visual stimulation feedback information, so as to complete the interaction. Search and switching can be performed among text input boxes, selection options and multiple directories, and a bottom layer of the directories can be reached, so as to select a target.

TECHNICAL FIELD

The present invention falls within the field of human-machineinteraction, and particularly relates to a human-machine interactionmethod based on visual stimulation.

BACKGROUND ART

Virtual reality (called VR for short), also referred to as LingJingtechnique, means an advanced human-machine computer interface havingessential features of immersion, interactivity and imagination. In thevirtual reality technology, the computer graphics, the emulationtechnique, the multimedia technology, the artificial intelligencetechnology, the technology of computer science, the concurrentprocessing technology and the multi-sensor technology are utilizedcomprehensively, to simulate functions of the sensory organs of humanssuch as vision, hearing and the tactile sense, so that people can beimmersed in a virtual world generated by a computer and can interactwith it in real time by natural means such as languages, gestures. Ahumanized multi-dimensional information space is created. A user notonly can feel, by a virtual reality system, immersive verisimilitudeexperienced in an objective physical world, but also can break throughthe limitations of space, time and the other objective limitations, tofeel experience that cannot be experienced by his/her own in the realworld.

AR is also referred to as augmented reality (called AR for short). It isa new technology seamlessly integrating real world information andvirtual world information, which simulates and superposes, by sciencetechnologies such as computers, entity information (vision information,sound, taste, tactile sense, etc.) can hardly be experienced within acertain time and space range of the real world. Virtual information isapplied to the real world and is sensed by human organs, achievingsensual experiences beyond the reality. In simple terms, VR is a fullyvirtual world, and AR is half-real and half-virtual world.

MR is also referred to as mixed reality (called MR for short) comprisesaugmented reality and augmented virtuality, and refers to a new visualenvironment produced by merging the reality world and a virtual world.In the new visual environment, physical objects and digital objectscoexist, and interact in real time.

The holographic projection technology is also referred to as a virtualimaging technology, and is a technology recording and reproducingthree-dimensional images with the principles of interference anddiffraction. The holographic projection technology not only can producea stereo overhead phantom, but also can enable the phantom to interactwith a performer, to complete a performance together, producing shockingperformance effects. The most iconic understanding about holographicprojection is “Jarvis” in “Iron Man”.

The simplest understanding of glass-free 3D is that the effect ofwatching a 3D movie with naked eyes is just as the effect of watching a3D movie using a pair of 3D glasses.

Immersive experience enables existing computer interaction tools such askeyboards, mouses and touchpads difficult to be used. How to develop aninteraction means more suitable for VR has become the focus of theindustry, and is still in the exploration and study stage.

Currently mainstream interaction means primarily include the following:

I. Sensory Immersion

In sensory immersion, interaction is primarily realized by collectingbody movements. The disadvantages is that the equipment is cumbersome(the collection of body movements are generally completed using themulti-camera technique) has complex structures, and many gesturecommands need to be memorized. The usage scenes are very limited.

II. Interactive Immersion

Interaction is primarily completed by motion tracking and key control.Generic produces include handles, joy sticks, body feeling guns,steering wheels, etc. Though facilities such as handles can realizeefficient control, problems of less keys, having a sole function and soon, especially that it needs to be held by a hand, affecting theimmersion. A more appropriate solution should be chosen.

Electrooculogram (EOG) is a bioelectrical signal produced by horizontalmotion, vertical motion, rotation or blink of eyeballs.

Electromyogram (EMG) is a bioelectrical signal produced by motions amuscle such as being static, contracting and being excited.

Electroencephalogram (EEG) is the overall reflection ofelectrophysiological activities of cranial nerve cells on cerebralcortex or the surface of scalp. In the aspect of program applications,people also try to realize a brain-computer interface (BCI) by utilizingelectroencephalogram signals, and to achieve some control purpose bymeans of effective extraction and classification of electroencephalogramsignals by utilizing the difference in people's electroencephalogram fordifferent sensations, motions or recognition activities.

SUMMARY OF THE INVENTION

Directed for the deficiency in the art, the present invention proposes ahuman-machine interaction method based on visual stimulation, andparticular technical solutions are as follows:

A human-machine interaction method based on visual stimulation, whichhuman-machine interaction method, as shown in FIG. 1, comprises 3 mainparts.

The human-machine interaction method of the present invention comprisesthe following steps:

Step 1: a software client is provided, wherein input ports of thesoftware client are connected to a human body physiological signalcollection device (in a wired or wireless manner), a user wears thecollection device, and communication ports of the client arerespectively connected to communication ports of a display module bymeans of a multichannel communication module.

The display module in step 1 comprises but is not limited to virtualreality, augmented reality, mixed reality and the other display modes.

The collection device in step 1 comprises but is not limited to varioustypes of apparatus for collecting and amplifying a human body signal.

The software client in step 1 is primarily responsible for accessing ahuman body physiological signal, human body signal recognition and aninteraction operation with the display module.

The so called human body physiological signal in step 1 comprises but isnot limited to an EEG, EOG, and EMG signal, which are referred to ashuman body physiological signals in collection.

Step 2: parameters are configured for a program in the display module,and if there are multiple graphic user interfaces (GUIs) in reality,then an ID number of a currently displayed GUI is passed back to aclient program, so as to provide a reference for signal recognition atthe client. and options for generating different functions at the GUIinterface are provided to a user for choosing, wherein each optioncorresponds to one virtual object or button, and the user completes acorresponding operation by selecting a needed virtual object or button(for example, typing some character, or implementing some menufunction).

Step 3: before a system starts to run, a training operation is performedat first, wherein the user gazes at a flickering target in a traininginterface (one button is generally used during training); acorresponding operation (blink is required for EOG, a finger movement isrequired for EMG, and gazing at a target is required for P300 in EEG) isperformed while the target is flickering, wherein the above-mentionedcollection device synchronously collects a human body bioelectricalsignal, performs band-pass filtering on the collected human bodybioelectrical signal, truncates a time period length of sampling points(generally 0-600 ms) from data, and performs down-sampling on thesampling points, the down-sampled data constructs a feature vector, anda computer stores a corresponding feature vector each time a virtual keyflickers; and a result obtained by averaging the feature vector is usedas a threshold, to establish a training model for the user, and makepreparation for a later recognition operation.

Step 4: the client completes the recognition and identification of atarget button (for example, changing the recognized button to red), andtransfers a result to the display module. The display module detectswhether an option corresponding to the target (a virtual object orbutton) is a subdirectory, if yes, then the subdirectory is entered, aninteraction interface is updated to an interaction interface of thesubdirectory, and processing is performed according to step 2.

Otherwise, a next step is entered.

Step 5: whether the option is a text input type is judged, if it is atext input type, then a next step is entered, otherwise, step 6 isentered.

Step 6: the interaction interface is switched to a text input interface,wherein selection characters in the text input interface are all virtualkeys, and after the user's character input is complete, a correspondingfunction is executed according to an input instruction.

Step 7: a function of the option is executed.

Further, the step 2 comprises the following steps:

Step 21: different options are generated in the GUI, each optioncorresponding one virtual object or button.

Step 22: the user gazes at a virtual object or button corresponding toan option needing to be selected.

Step 23: each virtual object or button flickers once in a certainsequence or randomly, when the virtual object or button needing to beselected by the user flickers, the target is determined with theoperations in step 3: the above-mentioned collection devicesynchronously collects a human body physiological signal, performsband-pass filtering on the collected human body physiological signal,truncates a time period length of sampling points (generally 0-600 ms)from data, performs down-sampling on the sampling points and performs awaveform detection operation, if a feature thereof passes the waveformdetection, then it is considered that a potential target exists, andtransformation is performed with a certain algorithm to construct afeature vector; and a computer stores a corresponding feature vectoreach time a virtual object or button flickers.

Step 24: the feature vector generated in step 23 is compared to atraining mode with a certain algorithm, if a similarity exceeds athreshold, then a decision is a target button, and a serial number ofthe target button is sent to the display module.

Step 25: the display module executes an operation corresponding to thebutton of the serial number.

Further, the step 6 comprises the following steps:

Step 51: the human-interface interaction interface is switched to a textinput interface, wherein selection characters in the text inputinterface are all virtual objects or buttons.

Step 52: the user gazes at selection characters to be selected.

Step 53: each virtual object or button flickers once in a certainsequence or randomly, and when the virtual object or button needing tobe selected by the user flickers, the target is determined with theoperations in step 3. The collection device of claim 1 synchronouslycollects a human body bioelectrical signal, performs band-pass filteringon the collected human body bioelectrical signal, truncates a timeperiod length of sampling points (generally 0-600 ms) from data,performs down-sampling on the sampling points and performs a waveformdetection operation, if a feature thereof passes the waveform detection,then it is considered that a potential target exists, and transformationis performed with a certain algorithm to construct a feature vector; anda computer stores a corresponding feature vector each time a virtualobject or button flickers.

Step 54: the feature vector generated in step 23 is compared to atraining mode with a certain algorithm, if a similarity exceeds athreshold, then a decision is a target button, and a serial number ofthe target button is sent to the display module.

Step 55: the display module executes a character corresponding to thebutton of the serial number, and completes character input.

Step 56: step 51 to step 55 are repeated, until text input is complete.

Step 57: a corresponding function is executed according to an inputinstruction.

The beneficial effects of the present invention are as follows:

In the first, the limitations of a sole user and a sole functioncurrently existing in the human-machine interaction method in VR, AR, MRand relevant fields are solved, and continuous interaction operationscan be performed with the interaction method.

In the second, search and switching can be performed among text inputboxes, selection options and multiple directories, and a bottom layer ofthe directories can be reached, so as to select a target. If a target isa text input target, an interface is switched to a text input interface,and texts are input by using human-body biological signal detection. Ifthe target is not a text input target, then an information type isdetermined by means of detection information of the human-bodybiological signal and visual feedback information, and a functionrepresented by the information is determined, so as to complete theinteraction.

In addition, with respect to the recognition method in the presentinvention, two methods i.e., waveform matching and a classifier are usedin combination, while ensuring an EMG signal or EOG signal is normallydetected, misjudgment of an input signal by a system is eliminated, andthe detection rate and the recognition degree are improved. When thepresent invention is used for a virtual keyboard, selection keys flickerin multiple rows; a finger EMG or EOG signal is judged by operating avirtual key in the row, to determine a target selection key. Thismatches a keyboard input method of an existing PC machine, and cangreatly improve the selection efficiency of functional keys.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of structural composition of ahuman-machine interaction system in the present invention;

FIG. 2 is a flow chart according to the present invention;

FIG. 3 is a structural diagram of a present virtual karaoke bar songchoosing system;

FIG. 4 is a schematic diagram of a brain-machine interaction interface;

FIG. 5 is a schematic diagram of a text input box;

FIG. 6 is a structural diagram of a virtual keyboard; and

FIG. 7 is a detection oscillogram of an EOG.

DETAILED DESCRIPTION OF EMBODIMENTS

Preferred embodiments of the present invention will be described belowin detail in conjunction with the accompanying drawings such that theadvantages and features of the present invention will be easier tounderstand by a person skilled in the art, so as to define the scope ofprotection of the present invention more clearly.

Embodiment 1

Particular embodiments of the present invention are as shown in FIGS. 2to 5, and a virtual reality karaoke bar song choosing system is taken asan example. A virtual reality multi-layer menu interaction method basedon a brain-machine interface, wherein the following steps are used inthe interaction method:

Step 1: a user wears an electroencephalogram acquisition cap, starts avirtual music song choosing system and enters a brain-machineinteraction interface, wherein the virtual music song choosing systemconsists of four levels of sub-systems.

Step 2: three options, i.e., choosing a song according to singers,choosing a song by pinyin and choosing a song by a song name, aregenerated in the brain-machine interaction interface, each optioncorresponding to a P300 virtual key.

Step 3: the user gazes at a P300 virtual key corresponding to an optionneeding to be selected, wherein it is assumed that the user is gazing ata P300 virtual key correspondingly related to the option of choosing asong by a song name herein.

Step 4: each P300 virtual key flickers once randomly, while the P300virtual key is flickering, the electroencephalogram acquisition capsynchronously collects a scalp electroencephalogram signal, performsband-pass filtering on the scalp electroencephalogram signal, truncatessampling points of 0-600 ms from electroencephalogram data, and performs1/6 down-sampling on the above-mentioned sampling points of 0-600 ms,the 1/6 down-sampled data constructs a feature vector, and a computerstores a corresponding feature vector when each P300 virtual keyflickers.

Step 5: the above-mentioned step 4 is repeated for 3 times, the computergenerates corresponding 3 feature vectors for all the P300 virtual keys,and the 3 feature vectors construct a feature vector set D1.

Step 6: the feature vector set D1 is classified by a classificationsystem, so as to determine a P300 virtual key selected by the user;here, the P300 virtual key corresponds to an option of choosing a songby a song name, in particular, each P300 virtual key super superposesthe sum of 3 feature vectors corresponding to 3 turns, and then averagessame, for example, the feature vector average of a first P300 virtualkey is D_(average):

$D_{average} = {d_{1} = \frac{d_{11}^{\prime} + d_{12}^{\prime} + d_{13}^{\prime}}{t}}$

D_(average) is solved for 3 turns of feature vectors corresponding toeach P300 virtual key respectively, to obtain a feature vector setD1=[d₁, d₂, d₃], and classification and waveform detection are performedon the feature vector set D1=[d₁, d₂, d₃], to obtain 3 classificationresults S=[s₁, s₂, s₃] and 3 waveform detection results W=[w₁, w₂, w₃],wherein only first 2 maximum scores are kept for the classificationresults S, and the remaining one is set to zero; the classificationresults S and the waveform detection results W are multiplied, to obtainR^(t):

R ^(t)=[r ₁ ^(t) ,r ₂ ^(t) ,r ₃ ^(t)]=[s ₁ ·w ₁ ,s ₂ ·w ₂ ,s ₃ ·w ₃]

R^(t) is a 3-dimensional row vector containing M none-zero values;

R^(t) is traversed, if there is no r_(i) ^(t)>0, then there is no targetoutput in a current turn, and step 3 is returned to, so as to go ondetection; and if there is an r_(i) ^(t)>0, then it is a target output.

Step 7: whether an option corresponding to the P300 virtual key is asubdirectory, and here the subdirectory is choosing a song by a songname; if yes, then the subdirectory is entered, and here thesubdirectory is turning a page of song names; and a brain-machineinteraction interface is updated to a brain-machine interactioninterface of turning a page of song names, and processing is performedaccording to step 3 to step 6;

otherwise, a next step is entered.

Step 8: whether the option is a text input type is judged, and it ischoosing a song according to singers and choosing a song by pinyin; ifit is choosing a song according to singers or choosing a song by pinyin,then a next step is entered, otherwise, step 16 is entered.

Step 9: the brain-interface interaction interface is switched to a textinput interface, wherein selection characters in the text inputinterface are all P300 virtual keys, and the selection charactersinclude 28 characters in total.

Step 10: the user gazes at selection characters to be selected.

Step 11: each selection character flickers once randomly, while theselection character is flickering, the electroencephalogram acquisitioncap synchronously collects a scalp electroencephalogram signal, performsband-pass filtering on the scalp electroencephalogram signal, truncatessampling points of 0-600 ms from electroencephalogram data, and performs1/6 down-sampling on the above-mentioned sampling points of 0-600 ms(selecting one from every 6 sampling points), the 1/6 down-sampled dataconstructs a feature vector, and a computer stores a correspondingfeature vector when each selection character flickers.

Step 12: the above-mentioned step 11 is repeated for 3 times, thecomputer generates corresponding 3 feature vectors for all the 28selection characters.

Step 13: the 3 feature vectors corresponding each selection character ofthe 28 characters are classified by a classification systemrespectively, so as to determine a character selected by the user, inparticular:

n=3 indicates 3 turns in total, each character superposes the sum of 3feature vectors corresponding to the 3 turns, and then averages same,for example, the feature vector average of a character is D_(average):

$D_{average} = {d_{1} = \frac{d_{11}^{\prime} + d_{12}^{\prime} + d_{12}^{\prime}}{t}}$

D_(average) is solved for 3 turns of feature vectors corresponding toeach character of the 28 characters respectively, to obtain a featurevector set D2=[d₁, d₂, . . . , d₂₈], and classification and waveformdetection are performed on the feature vector set D2=[d₁, d₂, . . . ,d₂₈], to obtain 28 classification results S=[s₁, s₂, . . . , s_(i), . .. , s₂₈] and 28 waveform detection results W=[w₁, w₂, . . . , w_(i), . .. , w₂₈], wherein only first 5 maximum scores are kept for theclassification results S, and the remaining are set to zero; theclassification results S and the waveform detection results W aremultiplied, to obtain R^(t):

R ^(t)=[r ₁ ^(t) ,r ₂ ^(t) , . . . ,r _(i) ^(t) . . . r ₂₈ ^(t)]=[s ₁ ·w₁ ,s ₂ ·w ₂ , . . . ,s _(i) ·w _(i) , . . . ,s _(N) ·w ₂₈]

R^(t) is a 28-dimensional row vector containing M none-zero values;

R^(t) is traversed, if there is no r_(i) ^(t)>0, then there is no targetoutput in a current turn, and step 10 is returned to, so as to go ondetection; and if there is an r_(i) ^(t)>0, then it is a target output.

Step 14: step 10 to step 13 are chosen repeated, until text input iscomplete, and here it is the completion of singer name input or thecompletion of pinyin input.

Step 15: a corresponding function is executed according to an inputinstruction.

Step 16: a music playing function is executed.

Embodiment II

The embodiment is an interaction method based on EOG and EMG, and thestructure of a virtual keyboard is as shown in FIG. 2.

the following steps are used in the method:

Step 1: a user wears an EOG collection apparatus, with a finger EMGcollection apparatus being respectively configured on 10 fingers, opensa virtual system and enters a brain-machine interaction interface.

Step 2: 4 selection keys are generated in the brain-machine interactioninterface and are represented by numbers 1, 2, 3 and 4 on the virtualkeyboard, each selection key corresponds to a row of virtual keys, therow of virtual keys consists of 10 virtual keys, 4 rows and 10 columnsof virtual keys are formed in total, the 10 column virtual keysrespectively correspond to the 10 finger EMG collection apparatus, theselection of the 4 selection keys is controlled by an EOG signal, andeach row of 10 key keys are controlled by a corresponding finger EMGsignal, wherein ten keys in each row from the left to right arerespectively manipulated by a left-hand little finger EMG, a left-handring finger EMG, a left-hand middle finger EMG, a left-hand index fingerEMG, a left-hand thumb EMG, a right-hand thumb RMG, a right-hand indexfinger EMG, a right-hand middle finger EMG, a right-hand ring finger EMGand a right-hand little finger EMG.

Step 3: the 4 selection keys flicker in turn, and only one of theselection keys flickers each time.

Step 4: when a selection key needing to be selected flickers, the usercarries out a corresponding eye movement to produce an EOG signal.

Step 5: a computer judges whether there is a selection key beingselected, if yes, a row of virtual keys corresponding to the selectedselection key is entered, and a next step is entered, otherwise, step 4is returned to.

Step 6: 10 virtual keys in the row of virtual keys flicker once in turn.

Step 7: when a selection key needing to be selected flickers, the user'sfinger corresponding to the virtual key makes a corresponding movementto produce a finger EMG signal, when each virtual key flickers, thecomputer takes a period of signal from the corresponding finger EOGcollection apparatus as a feature vector, and the 10 feature vectorsform a feature vector set Di.

Step 8: feature vector data in the feature vector set Di is successivelypreprocessed, including removing baseline drift, removing 50 Hzoperating frequency interference, and band-pass filtering.

Step 9: a first-order difference for each pieces of preprocessed vectorfeature data in the feature vector set Di is solved, and the particularmethod is: di=xi+1-xi

where i denotes an ith sampling point, d denotes a differenced signal,and x denotes a sampling value.

Step 10: for the feature vector set Di=[d1, d2, . . . , dN], the featurevectors in the feature vector set Di respectively correspond to theflicker of n selection keys in an operating row, and the feature vectorset Di is classified to obtain n classification results S=[s1, s2, . . ., si, . . . , sN], where only first 5 maximum scores are kept for theclassification results S, and the remaining are set to zero.

Step 11: waveform detection is performed on the feature vectors in thefeature vector set Di successively.

Step 12: whether a valley appears at a location 30-140 millisecondslater after a peak appears is judged, if yes, a next step is entered,otherwise, step 16 is entered.

Step 13: whether the peak/valley corresponds to a maximum/minimum pointof the entire period of signal is judged, if yes, a next step isentered, otherwise, step 16 is entered.

Step 14: whether the total energy of the period of signal from the peakto the valley is greater than a preset threshold P is judged, if yes,then a next step is entered, otherwise, step 16 is entered.

Step 15: if the waveform detection of the feature vector is passed, thencorresponding wi=1.

Step 16: if the waveform detection of the feature vector is not passed,then corresponding wi=0.

Step 17: Wi=[w1, w2, . . . , wi, . . . , wN] is obtained.

Step 18: the classification results S and the waveform detection resultsW are multiplied, to obtain Ri=[r1, r2, . . . , ri, . . . , rN].

Step 19: step 6 and step 18 are circulated for at least one round, toobtain a feature vector set Di+1 and a corresponding Ri+1.

Step 21: whether elements in Ri and Ri+1 are greater than zero isjudged, if yes, then a next step is entered, otherwise, step 6 isreturned to.

Step 22: whether elements in Ri have the same locations, sequence andsize as those in Ri+1 is judged, if yes, a next step is entered,otherwise, step 6 is returned to.

Step 23: if the feature vectors in the feature vector set Di are thesame as those in the feature vector set Di+1, a virtual key isdetermined according to a target feature vector in the feature vectorset Di and the feature vector set Di+1.

Step 24: whether a quit button is pressed is judged, if yes, step 28 isentered, otherwise, a next step is entered, wherein the judgment methodis particularly: a quit button is configured on the human-machineinteraction interface, and the quit button always keep flickeringcyclically as the numbers 1, 2, 3 and 4 on the virtual keyboard, and bydetecting whether a corresponding EOG signal is produced when the quitbutton flickers, whether the quit button is pressed is judged.

Step 25: whether input is complete is judged, if input is complete, step27 is entered, otherwise, a next step is entered, particularly, aconfirm button is configured on the human-machine interaction interface,and the confirm button always keep flickering cyclically as the numbers1, 2, 3 and 4 on the virtual keyboard and as the quit button, and bydetecting whether a corresponding EOG signal is produced when theconfirm button flickers, whether the quit button is pressed is judged.

Step 26: step 4 to step 22 are repeated.

Step 27: then execution is implemented according to the input virtualkey.

Step 28: selection ends.

The particular flow of electrooculogram (EOG) signal detection in step 3to step 5 mentioned above comprises:

Step 1: the m selection keys flicker in turn, and only one of theselection keys flickers each time.

Step 2: when a selection key needing to be selected flickers, the usercarries out a corresponding eye movement to produce an EOG signal, eachtime the selection key flickers, the EOG collection apparatus takes aperiod of signal as a feature vector, and m feature vectors compose afeature vector set Mi.

Step 3: feature vector data in the feature vector set Mi is successivelypreprocessed, including removing baseline drift, removing 50 Hzoperating frequency interference, and band-pass filtering.

Step 4: a first-order difference for each pieces of preprocessed vectorfeature data in the feature vector set Mi is solved, and the particularmethod is: di=xi+1-xi

where i denotes an ith sampling point, d denotes a differenced signal,and x denotes a sampling value.

Step 5: for the feature vector set Mi=[m1, m2, . . . , mm], the featurevectors in the feature vector set Mi respectively correspond to theflicker of m selection keys in an operating row, and the feature vectorset Mi is classified to obtain m classification results S=[s1, s2, . . ., si, . . . , sm], where only first q maximum scores are kept for theclassification results S, and the remaining are set to zero.

Step 6: waveform detection is performed on the feature vectors in thefeature vector set Mi successively.

Step 7: whether a valley appears at a location 30-140 milliseconds laterafter a peak appears is judged, if yes, a next step is entered,otherwise, step 16 is entered.

Step 8: whether the peak/valley corresponds to a maximum/minimum pointof the entire period of signal is judged, if yes, a next step isentered, otherwise, step 16 is entered.

Step 9: whether the total energy of the period of signal from the peakto the valley is greater than a preset threshold P is judged, if yes,then a next step is entered, otherwise, step 16 is entered.

Step 10: if the waveform detection of the feature vector is passed, thencorresponding wi=1.

Step 11: if the waveform detection of the feature vector is not passed,then corresponding wi=0.

Step 12: Wi=[w1, w2, . . . , wi, . . . , wm] is obtained.

Step 13: the classification results S and the waveform detection resultsW are multiplied, to obtain Ri=[r1, r2, . . . , ri, . . . , rN].

Step 14: whether elements in Ri are greater than zero is judged, if yes,then a next step is entered, otherwise, step 2 is returned to.

Step 15: step 6 and step 18 are circulated for at least one round, toobtain a feature vector set Mi+1 and a corresponding Ri+1.

Step 16: whether elements in Ri have the same locations, sequence andsize as those in Ri+1 is judged, if yes, a next step is entered,otherwise, step 2 is returned to.

Step 17: a row of virtual keys corresponding to the selected selectionkey is entered, and the finger EMG apparatus respectively corresponds tothe n virtual keys in the row.

Embodiment 3

This embodiment is an interaction method based on EMG, and particularexplanation is made by taking a virtual keyboard as an example inconjunction with FIGS. 2, 6 and 7:

this embodiment is as shown in FIGS. 2, 6 and 7, and the operation of avirtual keyboard is taken as an example.

An interaction method based on EMG, characterized in that

the following steps are used:

Step 1: a user connects 10 fingers respectively to 10 signal collectionends of an EMG collection apparatus, opens a virtual system and enters abrain-machine interaction interface.

Step 2: 4 rows of selection keys and 2 rows of buttons are generated ina brain-machine interaction interface, wherein the 2 rows of buttons arerespectively a quit row button and a confirm row button, each row of theother 4 rows of selection keys is provided with 10 selection keys, aunique number mark is set for each selection key, and the 10 columns ofselection keys in the 4 selection keys respectively correspond to 10signal collection ends on a one-to-one basis;

ten columns of keys from the left to right are respectively manipulatedby a left-hand little finger EMG, a left-hand ring finger EMG, aleft-hand middle finger EMG, a left-hand index finger EMG, a left-handthumb EMG, a right-hand thumb RMG, a right-hand index finger EMG, aright-hand middle finger EMG, a right-hand ring finger EMG and aright-hand little finger EMG;

and one row of the 4 rows of selection keys are selected, then a columnof selection keys are selected therefrom, thus a unique number mark canbe determined. The quit row button and the confirm row button are alsoconfigured with a unique mark; for the quit button, an EMG signal can beinput through any one of 10 signal collection ends, here a left-handindex finger EMG is used for signal input; and for the confirm rowbutton, an EMG signal can also be input through any one of 10 signalcollection ends, here a right-hand index EMG is used for signal input.

Step 3: the 4 rows of selection keys and the 2 rows of buttons flickerin turn row by row, and when the 4 rows of selection key flicker, anentire row of selection keys simultaneously flicker each time.

Step 4: whether an EMG signal is produced when the entire row flickersis judged, if not, step 3 is returned to, so a next entire row flickers;if yes, then the row is assumed to be an operating row, and step 5 isentered.

Step 5: if the operating row is one of 4 rows of selection keys, the 10columns of selection keys in the operating row flicker in sequence, wheneach selection key flickers, a marking system sends a number markcorresponding to the selection key and inserts same to a sampling timeaxis of the EMG collection apparatus.

Step 6: when a selection key needing to be selected flickers, the usermakes a corresponding movement with a finger corresponding to theselection key.

Step 7: the EMG collection apparatus takes a period of signal from thesignal collection end corresponding to the number mark as a featurevector, and generates 10 feature vectors when the flickering of the 10keys are all completed, and the 10 feature vectors form a feature vectorset Di.

Step 8: feature vector data in the feature vector set Di is successivelypreprocessed, including removing baseline drift, removing 50 Hzoperating frequency interference, and band-pass filtering.

Step 9: a first-order difference for each pieces of preprocessed vectorfeature data in the feature vector set Di is solved, and the particularmethod is:

di=xi+1−xi, where i denotes an ith sampling point, d denotes adifferenced signal, and x denotes a sampling value.

Step 10: the differenced feature vector set is Di=[d1, d2, . . . , d10],the feature vectors in the feature vector set Di respectively correspondto the flicker of n selection keys in an operating row, and the featurevector set Di is classified to obtain n classification results S=[s1,s2, . . . , si, . . . , s10], where only first 3 maximum scores are keptfor the classification results S, and the remaining are set to zero.

Step 11: waveform detection is performed on the feature vectors in thefeature vector set Di successively.

Step 12: whether a valley appears at a location 30-140 millisecondslater after a peak appears is judged, if yes, a next step is entered,otherwise, step 16 is entered.

Step 13: whether the peak/valley corresponds to a maximum/minimum pointof the entire period of signal is judged, if yes, a next step isentered, otherwise, step 16 is entered.

Step 14: whether the total energy of the period of signal from the peakto the valley is greater than a preset threshold P is judged, if yes,then a next step is entered, otherwise, step 16 is entered.

Step 15: if the waveform detection of the feature vector is passed, thencorresponding wi=1.

Step 16: if the waveform detection of the feature vector is not passed,then corresponding wi=0.

Step 17: waveform detection results W=[w1+w2+wi . . . +w10] are obtainedby step 15 and step 16, and the classification results S and thewaveform detection results W are multiplied, to obtain Ri=[r1, r2, . . ., ri, . . . , r10].

Step 18: Ri is traversed, if there is no ri>0, then determining thatthere is no target feature vector in the feature vector set Di, andentering step 22, and if there is an ri>0, then determining that thereis a target feature vector in the feature vector set Di, and entering anext step.

Step 19: step 5 and step 17 are circulated for at least one round, toobtain a feature vector set Di+1 and a corresponding Ri+1.

Step 20: elements at corresponding locations in Ri and Ri+1 are comparedto see whether they are the same, if they are the same, then whether thefeature vector set Di and the feature vector set Di+1 contain the samefeature vector is judged, a selection key is determined according to thetarget feature vector in the feature vector set Di and the featurevector set Di+1, the selection of an entire row is quit to, and a nextstep is entered, otherwise, step 5 is returned to.

Step 21: when the quit row button is in a flickering state, whether thequit row button is pressed is judged, if yes, step 25 is entered,otherwise, a next step is entered.

Step 22: when the confirm row button is in a flickering state, whetherthe confirm row button is pressed is judged, if yes, step 24 is entered,otherwise, a next step is entered.

Step 23: a next operating row is entered, and step 4 to step 22 arerepeated.

Step 24: then execution is implemented according to an input selectionoption.

Step 25: selection ends.

1. A human-machine interaction method based on visual stimulation,characterized in that the human-machine interaction method comprises thefollowing steps: step 1: a software client is provided, wherein inputports of the software client are connected to a human body physiologicalsignal collection device in a wired or wireless manner, a user wears thecollection device, and communication ports of the client arerespectively connected to communication ports of a display module bymeans of a multichannel communication module; the display module is usedfor virtual reality (VR), augmented reality (AR), mixed reality (MR),holographic projection or glass-free 3D; the collection device is anapparatus for collecting and amplifying a human body signal; thesoftware client is used for accessing a human body physiological signal,relevant algorithm detection for human body signal recognition and aninteraction operation with the display module; and the human bodyphysiological signal is an EEG, EOG, or EMG signal; step 2: parametersare configured for a program in the display module, and if there aremultiple graphic user interfaces (GUIs) in reality, then an ID number ofa currently displayed GUI is passed back to a client program, so as toprovide a reference for signal recognition at the client; and optionsfor generating different functions at the GUI interface are provided toa user for choosing, wherein each option corresponds to one virtualobject or button, and the user completes a corresponding operation byselecting a needed virtual object or button; step 3: before a systemstarts to run, a training operation is performed at first, wherein theuser gazes at a flickering target in a training interface, for visualstimulation; a corresponding operation is performed while the target isflickering, wherein the collection device synchronously collects a humanbody bioelectrical signal, performs band-pass filtering on the collectedhuman body bioelectrical signal, truncates a time period length ofsampling points from data, and performs down-sampling on the samplingpoints, the down-sampled data constructs a feature vector, and acomputer stores a corresponding feature vector each time a virtual keyflickers; and a result obtained by averaging the feature vector is usedas a threshold, to establish a training model for the user, and makepreparation for a later recognition operation; step 4: the clientcompletes the recognition and identification of a target button, andtransfers a result to the display module; the display module detectswhether an option corresponding to the target is a subdirectory, if yes,then the subdirectory is entered, an interaction interface is updated toan interaction interface of the subdirectory, and processing isperformed according to step 2; otherwise, a next step is entered; step5: whether the option is a text input type is judged, if it is a textinput type, then a next step is entered, otherwise, step 6 is entered;step 6: the interaction interface is switched to a text input interface,wherein selection characters in the text input interface are all virtualkeys, and after the user's character input is complete, a correspondingfunction is executed according to an input instruction; and step 7: afunction of the option is executed.
 2. The human-machine interactionmethod based on visual stimulation according to claim 1, characterizedin that the step 2 comprises the following steps: step 21: differentoptions are generated in the GUI, each option corresponding one virtualobject or button; step 22: the user gazes at a virtual object or buttoncorresponding to an option needing to be selected; step 23: each virtualobject or button flickers once in a certain sequence or randomly, whenthe virtual object or button needing to be selected by the userflickers, the target is determined with the operations in step 3: thecollection device synchronously collects a human body physiologicalsignal, performs band-pass filtering on the collected human bodyphysiological signal, truncates a time period length of sampling pointsfrom data, performs down-sampling on the sampling points and performs awaveform detection operation, if a feature thereof passes the waveformdetection, then it is considered that a potential target exists, andtransformation is performed with a certain algorithm to construct afeature vector; and a computer stores a corresponding feature vectoreach time a virtual object or button flickers; step 24: the featurevector generated in step 23 is compared to a training mode with acertain algorithm, if a similarity exceeds a threshold, then a decisionis a target button, and a serial number of the target button is sent tothe display module; and step 25: the display module executes anoperation corresponding to the button of the serial number.
 3. Thehuman-machine interaction method based on visual stimulation accordingto claim 1, characterized in that the step 6 comprises the followingsteps: step 51: the human-interface interaction interface is switched toa text input interface, wherein selection characters in the text inputinterface are all virtual objects or buttons; step 52: the user gazes atselection characters to be selected; step 53: each virtual object orbutton flickers once in a certain sequence or randomly, when the virtualobject or button needing to be selected by the user flickers, the targetis determined with the operations in step 3: the collection devicesynchronously collects a human body bioelectrical signal, performsband-pass filtering on the collected human body bioelectrical signal,truncates a time period length of sampling points from data, performsdown-sampling on the sampling points and performs a waveform detectionoperation, if a feature thereof passes the waveform detection, then itis considered that a potential target exists, and transformation isperformed with a certain algorithm to construct a feature vector; and acomputer stores a corresponding feature vector each time a virtualobject or button flickers; step 54: the feature vector generated in step23 is compared to a training mode with a certain algorithm, if asimilarity exceeds a threshold, then a decision is a target button, anda serial number of the target button is sent to the display module; step55: the display module executes a character corresponding to the buttonof the serial number, and completes character input; step 56: step 51 tostep 55 are repeated, until text input is complete; and step 57: acorresponding function is executed according to an input instruction. 4.A virtual reality multi-layer menu interaction method based on abrain-machine interface, characterized in that the following steps areused in the interaction method: step 1: a user wears anelectroencephalogram acquisition cap, starts a virtual reality systemand enters a brain-machine interaction interface; step 2: differentoptions are generated in the brain-machine interaction interface, eachoption corresponding to a P300 virtual key, and the user selects aneeded P300 virtual key; step 3: whether an option corresponding to theneeded P300 virtual key is a subdirectory, if yes, then the subdirectoryis entered, a brain-machine interaction interface is updated to abrain-machine interaction interface of the subdirectory, and processingis performed according to step 2; otherwise, a next step is entered;step 4: whether the option is a text input type is judged, if it is atext input type, then a next step is entered, otherwise, step 6 isentered; step 5: the brain-machine interaction interface is switched toa text input interface, wherein selection characters in the text inputinterface are all P300 virtual keys, and after the user's characterinput is complete, a corresponding function is executed according to aninput instruction; and step 6: a function of the option is executed. 5.The virtual reality multi-layer menu interaction method based on abrain-machine interface according to claim 4, characterized in that thestep 2 comprises the following steps: step 21: different options aregenerated in the brain-machine interaction interface, each optioncorresponding to a P300 virtual key; step 22: the user gazes at a P300virtual key corresponding to an option needing to be selected; step 23:each P300 virtual key flickers once randomly, while the P300 virtual keyis flickering, the electroencephalogram acquisition cap synchronouslycollects a scalp electroencephalogram signal, performs band-passfiltering on the scalp electroencephalogram signal, truncates samplingpoints of 0-600 ms from electroencephalogram data, and performs 1/6down-sampling on the sampling points of 0-600 ms, the 1/6 down-sampleddata constructs a feature vector, and a computer stores a correspondingfeature vector when each P300 virtual key flickers; step 24: theabove-mentioned step 22 to step 23 are repeated for n times, thecomputer generates corresponding N feature vectors for all the P300virtual keys, and the N feature vectors construct a feature vector setD1; and step 25: the feature vector set D1 is classified by aclassification system, so as to determine a P300 virtual key selected bythe user.
 6. The virtual reality multi-layer menu interaction methodbased on a brain-machine interface according to claim 4, characterizedin that the step 5 comprises the following steps: step 51: thebrain-interface interaction interface is switched to a text inputinterface, wherein selection characters in the text input interface areall P300 virtual keys; step 52: the user gazes at selection charactersto be selected; step 53: each selection character flickers oncerandomly, while the selection character is flickering, theelectroencephalogram acquisition cap synchronously collects a scalpelectroencephalogram signal, performs band-pass filtering on the scalpelectroencephalogram signal, truncates sampling points of 0-600 ms fromelectroencephalogram data, and performs 1/6 down-sampling on thesampling points of 0-600 ms, the 1/6 down-sampled data constructs afeature vector, and a computer stores a corresponding feature vectorwhen each selection character flickers; step 54: the above-mentionedstep 52 to step 53 are repeated for n times, the computer generatescorresponding N feature vectors for all the selection characters, andthe N feature vectors construct a feature vector set D2; step 55: thefeature vector set D2 is classified by a classification system, so as todetermine a character selected by the user; step 56: step 52 to step 55are repeated, until text input is complete; and step 57: a correspondingfunction is executed according to an input instruction.
 7. Aninteraction method based on EOG and EMG, characterized in that thefollowing steps are used: step 1: a user wears an EOG collectionapparatus, with a finger EMG collection apparatus being configured on atleast n fingers, opens a system and enters a brain-machine interactioninterface; step 2: m selection keys are generated in the brain-machineinteraction interface, each selection key corresponds to a row ofvirtual keys, the row of virtual keys consists of n virtual keys, m rowsand n columns of virtual keys are formed in total, the n column virtualkeys respectively correspond to the n finger EMG collection apparatus,the selection of the m selection keys is controlled by an EOG signal,and each row of n key keys are controlled by a corresponding finger EMGsignal; step 3: the m selection keys flicker in turn, and only one ofthe selection keys flickers each time; step 4: when a selection keyneeding to be selected flickers, the user carries out a correspondingeye movement to produce an EOG signal; step 5: a computer judges whetherthere is a selection key being selected, if yes, a row of virtual keyscorresponding to the selected selection key is entered, and a next stepis entered, otherwise, step 4 is returned to; step 6: n virtual keys inthe row of virtual keys flicker once in turn; step 7: when a selectionkey needing to be selected flickers, the user's finger corresponding tothe virtual key makes a corresponding movement to produce a finger EMGsignal, when each virtual key flickers, the computer takes a period ofsignal from the corresponding finger EOG collection apparatus as afeature vector, and the n feature vectors form a feature vector set Di;step 8: the feature vectors in the feature vector set Di arepreprocessed, and the feature vectors are judged by a classifier; step9: step 4 to step 8 are circulated for at least one round, to obtain afeature vector set Di+1; step 10: whether the feature vectors in thefeature vector set Di are the same as those in the feature vector setDi+1 is judged, if all of them are the same, a virtual key is determinedaccording to the target feature vectors in the feature vector set Di andthe feature vector set Di+1, and a next step is entered, otherwise, step6 is returned to; step 11: whether a quit button is pressed is judged,if yes, step 15 is entered, otherwise, a next step is entered; step 12:whether input is complete is judged, if input is complete, step 14 isentered, otherwise, a next step is entered; step 13: step 4 to step 12are repeated; step 14: then execution is implemented according to theinput virtual key; and step 15: selection ends.
 8. The interactionmethod based on EOG and EMG according to claim 7, characterized in thatthe step 8 comprises the following steps: step 81: feature vector datain the feature vector set Di is successively preprocessed, includingremoving baseline drift, removing 50 Hz operating frequencyinterference, and band-pass filtering; step 82: a first-order differencefor each pieces of preprocessed vector feature data in the featurevector set Di is solved, and the particular method is: di=xi+1−xi, wherei denotes an ith sampling point, d denotes a differenced signal, and xdenotes a sampling value; step 83: for the feature vector set Di=[d1,d2, . . . , dN], the feature vectors in the feature vector set Direspectively correspond to the flicker of n selection keys in anoperating row, and the feature vector set Di is classified to obtain nclassification results S=[s1, s2, . . . , si, . . . , sN], where onlyfirst q maximum scores are kept for the classification results S, andthe remaining are set to zero; step 84: waveform detection is performedon the feature vectors in the feature vector set Di, to obtain nwaveform detection results Wi=[w1, w2, . . . , wi, . . . , wN]; and step85: the classification results S and the waveform detection results Ware multiplied, to obtain Ri=[r1, r2, . . . , ri, . . . , rN].
 9. Theinteraction method based on EOG and EMG according to claim 7,characterized in that the step 5 comprises the following steps: step 91:step 4 and step 7 are circulated for at least one round, to obtain afeature vector set Di+1 and a corresponding Ri+1; and step 92: elementsat corresponding locations in Ri and Ri+1 are compared to see whetherthey are the same, if they are the same, then whether the feature vectorset Di and the feature vector set Di+1 contain the same feature vectoris judged, a selection key is determined according to the target featurevector in the feature vector set Di and the feature vector set Di+1, anda next step is entered, otherwise, step 4 is returned to.
 10. Theinteraction method based on EOG and EMG according to claim 7,characterized in that the step 84 comprises the following steps: step841: waveform detection is performed on the feature vectors in thefeature vector set Di successively; step 842: whether a valley appearsat a location 30-140 milliseconds later after a peak appears is judged,if yes, a next step is entered, otherwise, step 846 is entered; step843: whether the peak/valley corresponds to a maximum/minimum point ofthe entire period of signal is judged, if yes, a next step is entered,otherwise, step 846 is entered; step 844: whether the total energy ofthe period of signal from the peak to the valley is greater than apreset threshold P is judged, if yes, then a next step is entered,otherwise, step 846 is entered; step 845: if the waveform detection ofthe feature vector is passed, then corresponding wi=1; step 846: if thewaveform detection of the feature vector is not passed, thencorresponding wi=0; and step 847: Wi=[w1, w2, . . . , wi, . . . , wN] isobtained.
 11. The interaction method based on EOG and EMG according toclaim 7, characterized in that the step 10 comprises the followingsteps: step 101: whether elements in Ri and Ri+1 are greater than zerois judged, if yes, then a next step is entered, otherwise, step 6 isreturned to; step 102: whether elements in Ri have the same locations,sequence and size as those in Ri+1 is judged, if yes, a next step isentered, otherwise, step 6 is returned to; and step 103: if the featurevectors in the feature vector set Di are the same as those in thefeature vector set Di+1, a virtual key is determined according to atarget feature vector in the feature vector set Di and the featurevector set Di+1.
 12. An interaction method based on EMG, characterizedin that the following steps are used: step 1: a user connects n fingersrespectively to n signal collection ends of an EMG collection apparatus,opens a virtual system and enters a brain-machine interaction interface;step 2: m rows and n columns of selection keys are generated in abrain-machine interaction interface, a unique number mark is set foreach selection key, and the n columns of selection keys respectivelycorrespond to n signal collection ends on a one-to-one basis; step 3:the m rows of selection keys flicker in turn row by row, and an entirerow of selection keys simultaneously flicker each time; step 4: whetheran EMG signal is produced when the entire row of selection keys flickeris judged, if not, step 3 is returned to, so a next entire row ofselection keys flicker; if yes, then the row is assumed to be anoperating row, and step 5 is entered; step 5: the n columns of selectionkeys in the operating row flicker in sequence, when each selection keyflickers, a marking system sends a number mark corresponding to theselection key and inserts same to a sampling time axis of the EMGcollection apparatus; step 6: when a selection key needing to beselected flickers, the user makes a corresponding movement with a fingercorresponding to the selection key; step 7: the EMG collection apparatustakes a period of signal from the signal collection end corresponding tothe number mark as a feature vector, and generates n feature vectorswhen the flickering of the n keys are all completed, and the n featurevectors form a feature vector set Di; step 8: the feature vectors in thefeature vector set Di are preprocessed, and the feature vectors arejudged by a classifier; step 9: whether a target feature vector existsin the feature vector set Di, if the target feature vector exists, thena next step is entered, otherwise, step 12 is entered; step 10: step 5to step 9 are circulated for at least one round, to obtain a featurevector set Di+1, whether the feature vectors in the feature vector setDi are the same as those in the feature vector set Di+1 is judged, ifall of them are the same, a selection key is determined according to thetarget feature vectors in the feature vector set Di and the featurevector set Di+1, and a next step is entered, otherwise, step 5 isreturned to; step 11: whether the selection key indicates to end outputis judged, if yes, step 15 is entered, otherwise, a next step isentered; step 12: whether input is complete is judged, if input iscomplete, step 14 is entered, otherwise, a next step is entered; step13: a next operating row is entered, and processing is executed in thesequence of step 4 to step 12; step 14: then execution is implementedaccording to an input selection option; and step 15: selection ends. 13.The interaction method based on EMG according to claim 12, characterizedin that the step 8 comprises the following steps: step 81: featurevector data in the feature vector set Di is successively preprocessed,including removing baseline drift, removing 50 Hz operating frequencyinterference, and band-pass filtering; step 82: a first-order differencefor each pieces of preprocessed vector feature data in the featurevector set Di is solved, and the particular method is: di=xi+1-xi, wherei denotes an ith sampling point, d denotes a differenced signal, and xdenotes a sampling value; step 83: for the feature vector set Di=[d1,d2, . . . , dN], the feature vectors in the feature vector set Direspectively correspond to the flicker of n selection keys in anoperating row, and the feature vector set Di is classified to obtain nclassification results S=[s1, s2, . . . , si, . . . , sN], where onlyfirst q maximum scores are kept for the classification results S, andthe remaining are set to zero; step 84: waveform detection is performedon the feature vectors in the feature vector set Di, to obtain nwaveform detection results Wi=[w1, w2, . . . , wi, . . . , wN]; and step85: the classification results S and the waveform detection results Ware multiplied, to obtain Ri=[r1, r2, . . . , ri, . . . , rN]; and step9 particularly comprises: traversing Ri, if there is no ri>0, thendetermining that there is no target feature vector in the feature vectorset Di, and entering step 12, and if there is an ri>0, then determiningthat there is a target feature vector in the feature vector set Di, andentering a next step.
 14. The interaction method based on EMG accordingto claim 12, characterized in that the step 9 comprises the followingsteps: step 101: step 5 to step 9 are circulated for at least one round,to obtain a feature vector set Di+1 and a corresponding Ri+1; and step102: elements at corresponding locations in Ri and Ri+1 are compared tosee whether they are the same, if they are the same, then whether thefeature vector set Di and the feature vector set Di+1 contain the samefeature vector is judged, a selection key is determined according to thetarget feature vector in the feature vector set Di and the featurevector set Di+1, and a next step is entered, otherwise, step 5 isreturned to.
 15. The interaction method based on EMG according to claim12, characterized in that the step 84 comprises the following steps:step 841: waveform detection is performed on the feature vectors in thefeature vector set Di successively; step 842: whether a valley appearsat a location 30-140 milliseconds later after a peak appears is judged,if yes, a next step is entered, otherwise, step 846 is entered; step843: whether the peak/valley corresponds to a maximum/minimum point ofthe entire period of signal is judged, if yes, a next step is entered,otherwise, step 846 is entered; step 844: whether the total energy ofthe period of signal from the peak to the valley is greater than apreset threshold P is judged, if yes, then a next step is entered,otherwise, step 846 is entered; step 845: if the waveform detection ofthe feature vector is passed, then corresponding wi=1; step 846: if thewaveform detection of the feature vector is not passed, thencorresponding wi=0; and step 847: Wi=[w1, w2, . . . , wi, . . . , wN] isobtained.